This patent application claims priority based on a Japanese patent application, H11-184153 filed on Jun. 29, 1999 and H11-184163 filed on Jun. 29, 1999, the contents of which are incorporated herein by reference.
1. Field of the Invention
The present invention relates to stereoscopic imaging apparatus and camera and method therefor which pick up a stereoscopic image when an object is viewed from different points. More particularly, the present invention relates to the stereoscopic imaging apparatus and camera which image simultaneously a plurality of stereoscopic images.
2. Description of the Related Art
In the field of image processing as well as image recognition, generally adopted as a method to extract data of an object in the depth direction is a method in which a stereoscopic amount between stereoscopic images is detected utilizing a plurality of stereoscopic images obtained when the object is viewed from different points, so that the depth level up to the object is calculated from the stereoscopic amount.
As conventionally available apparatus which images a plurality of stereoscopic images, there is stereoscopic imaging apparatus shown in FIG. 1. The stereoscopic imaging apparatus includes a light passing portion 5 which includes an opening that passes the light on a pupil plane of the optical lens 4. The opening is moved in the parallel direction to the optical lens 4, and the image having passed the opening is imaged on a light receiving unit, so that a plurality of the stereoscopic images are imaged in sequence (see Japanese Patent Application Laid-Open No. Hei 10-42314).
When the opening is situated at 6a shown in FIG. 1, an image 3c in which an object 3 is focused is formed in a position away from the light receiving unit 7, so that an out-of-focus image 3a is imaged on the light receiving unit 7. When the opening is moved to a position 6b shown in FIG. 1, the image 3c in which the object 3 is focused is formed in the same position. However, in the light receiving unit 7 the out-of-focus image 3b is imaged on a position different from 3a. A displacement between the image imaged in the light receiving unit 7 and 3b is called a stereoscopic amount. By measuring the stereoscopic amount, a distance from the optical lens 4 to the object 3 can be calculated using a lens formula since both the distance between a position 6a and a position 6b in the opening unit and a focal length are known. By detecting the stereoscopic amount about all regions of the object and then calculating the distances between the optical lens 4 and the all regions, the distance distribution can be obtained.
Moreover, as another method of imaging the stereoscopic image, a plurality of openings are provided in the light passing portion 5, and one of the openings is opened with other openings being closed in a predetermined time interval, so that a plurality of the stereoscopic images can be imaged in sequence (see Japanese Patent Application Laid-Open No. H10-271534).
In order to obtain the depth of the object, the stereoscopic amount needs to be detected by comparing the images among the stereoscopic images. However, when a plurality of stereoscopic images are imaged in the above manner, a time lag occurs between when the first stereoscopic image is taken and when the second stereoscopic image is taken. Thus, if a camera is held by a hand, an error due to such unstable holding may result. Moreover, the object may move during this time lag. Thus, the error will result inevitably in a plurality of the stereoscopic images taken, and is problematic in that information on the correct depth of the object can not be obtained.
Therefore, it is an object of the present invention to provide stereoscopic imaging apparatus and method which overcome the above issues in the related art. This object is achieved by combinations described in the independent claims. The dependent claims define further advantageous and exemplary combinations of the present invention.
According to an aspect of the present invention, there is provided a stereoscopic imaging apparatus for imaging a plurality of stereoscopic images obtained when an object is viewed from different points, the apparatus comprising: an optical image forming unit having a single optical axis, which image-forms the object; a light receiving unit in which a plurality of light receiving elements are arranged and the object is image-formed by the optical image forming unit; alight passing portion including first and second openings through which light having passed through first and second regions in the optical image forming unit is irradiated to the light receiving unit; and an optical separation portion which simultaneously images both a first image having passed through the first opening and a second image having passed through the second opening, on the different light receiving elements of the light receiving unit.
Preferably, the first and second openings of the light passing portion are equipped with first and second opening optical filters which transmit different optical light components, respectively.
The optical separation portion may includes: a first light receiving portion optical filter which transmits again a light component having transmitted through the first opening optical filter; and a second light receiving portion optical filter which transmits again a light having transmitted through the second opening optical filter, wherein the first light receiving portion optical filter and the second light receiving optical filter are arranged in an alternate order.
The first and second opening optical filters of the light passing portion are, preferably, opening specific wavelength component transmitting filters which transmit different specific wavelength components of the light, respectively.
The optical separation portion may include: a first light receiving portion specific wavelength component transmitting filter which transmits the light having the same wavelength component as the first opening specific wavelength transmitting filter of the light passing portion; and a second light receiving portion specific wavelength component transmitting filter which transmits the light having the same wavelength component as the second opening specific wavelength component transmitting filter of the light passing portion, wherein the light receiving unit is covered in a manner such that the first and second light receiving portion specific wavelength component transmitting filters are arranged in alternate order.
Moreover, the first light receiving portion specific wavelength component transmitting filter may include: filters that transmit specific RGB wavelength components (xcexR1, xcexG1 and xcexB1), respectively, and the second light receiving portion specific wavelength component transmitting filter may include: filters that transmit specific RGB wavelength components (xcexR2, xcexG2 and xcexB2), respectively, and wherein the filter transmitting the wavelength xcexR1 and the filter transmitting xcexR2 are arranged adjacently; the filter transmitting the wavelength xcexG1 and the filter transmitting xcexG2 are arranged adjacently; and the filter transmitting the wavelength xcexB1 and the filter transmitting xcexB2 are arranged adjacently.
Moreover, the first and second opening optical filter of the light passing portion are, preferably, opening specific polarization component transmitting filters that transmit the light having a polarization plane of the horizontal direction and a polarization plane of the vertical direction, respectively.
Moreover, the optical separation portion may include light receiving portion specific polarization component transmitting filters which transmit the light having polarization planes of the horizontal and vertical directions, and the light receiving portion specific polarization component transmitting filters which transmit the light having polarization planes of the horizontal and vertical directions are, preferably, arranged in an alternate manner so as to cover the light receiving unit.
Moreover, the optical separation portion may include a polarization component separating portion that separates the light having polarization planes of the horizontal and vertical directions, and the light receiving unit may include: a first light receiving plane which receives light having the horizontal-direction polarization plane separated by the polarization component separating portion; and a second light receiving plane which receives light having vertical-direction polarization plane separated by the polarization component separating portion.
The apparatus may further comprise an electrically separating portion which electrically separates the first image that is imaged by the light receiving unit from the second image that is imaged by the light receiving unit.
The light receiving element is preferably an charge-coupled device, and the light receiving unit preferably comprises a photoelectric conversion image element in which a plurality of charge-coupled devices are arranged.
According to another aspect of the present invention, there is provided a camera for acquiring data on a distance between the camera and an object, comprising: a first optical image forming unit having a single optical axis, which image-forms the object; a first light receiving unit in which a plurality of light receiving elements are arranged and the object is image-formed by the first optical image forming unit; a light passing portion including first and second openings through which light having passed through first and second regions in the first optical image forming unit is irradiated to the first light receiving unit; an optical separation portion which simultaneously images both a first image having passed through the first opening and a second image having passed through the second opening, on the different light receiving elements of the first light receiving unit; and a distance calculating unit which calculates a distance between the optical image forming unit and at least a point on the object, based on the first and second images.
The camera may further comprise: a second optical image forming unit which image forms the object; a second light receiving unit which image-forms the object by the second optical image forming unit; and a control unit which controls at least one of focusing and aperture of the optical image forming unit, and exposure time of the second light receiving unit, based on the distance calculated by the distance calculating unit.
Moreover, the camera may further comprise: a second light receiving unit which image-forms the object by the first optical image forming unit; and a control unit which controls at least one of focusing and aperture of the first optical image forming unit, and exposure time of the second light receiving unit, based on the distance calculated by the distance calculating unit.
Moreover, the camera may further comprise a drive unit which moves the light passing portion and the optical separation portion out of an optical path defined by a space such that the object is light-received by the first light receiving unit.
The light passing portion further may include a third opening which irradiates to the first light receiving unit the light having passed through a third region in the first optical image forming unit, so that while the optical serration portion opens the first and second openings and closes the third opening in the light passing portion, the first image having passed through the first opening and the second image having passed through the second opening are simultaneously imaged on the different light receiving elements, and also while the optical separation portion closes the first and second openings and opens the third opening in the light passing portion, the image having passed through the third opening is imaged on the light receiving element.
Preferably, the first,second and third openings in the light passing portion include an LCD shutter.
The distance calculating unit calculates a distance between the optical image forming unit and the object that is image-formed on a first pixel address of the first image, based on the first pixel address of the first image and a second pixel address in the second image of the object that is image-formed on the first image address.
Moreover, the camera may further comprise a recoding unit which records an image imaged by the second light receiving portion and the distance calculated by the distance calculating unit.
According to still another aspect of the present invention, there is provided a stereoscopic apparatus comprising: an optical image forming unit having a single optical axis, which image-forms the object; a light receiving unit in which a plurality of light receiving elements are arranged and the object is image-formed by the optical image forming unit; a light passing portion including first and second openings through which light having passed through first and second regions in the optical image forming unit is irradiated to the light receiving unit; an imaging unit which simultaneously images both a first image that image-forms the object after passing through the first opening and a second image that image-forms the object after passing through the second opening, on the light receiving unit; and a distance calculating unit which calculates a distance between the optical image forming unit and at least a point on the object, based on the image imaged on the light receiving unit.
The distance calculating unit preferably includes a stereoscopic amount detecting unit which detects a stereoscopic amount of the first image having passed through the first opening and the second image having passed through the second opening in terms of a specific region of an image imaged in the light receiving unit, so that the distance between the optical image forming unit and the object imaged at the specific region is calculated based on the stereoscopic amount.
The distance calculating unit may further include: an auto-correlation calculating unit which calculates correlation between the image of the specific region imaged in the light receiving unit and a reference image acquired by shifting the image in a stereoscopic direction, while a shifted amount thereof is varied by a predetermined amount, so that the stereoscopic detecting unit obtains the stereoscopic amount utilizing the correlation calculated by the auto-correlation calculating unit.
The distance calculating unit may further include: an edge extracting unit which extracts an edge image of the image imaged by the light receiving unit, whereby the auto-correlation calculating unit calculates correlation between the edge image of the specific region imaged by the light receiving unit and the reference image acquired by shifting the edge image in the stereoscopic direction while a shifted amount is varied by a predetermined amount.
The auto-correlation calculating unit calculates correlation between the image of the specific region imaged by the light receiving unit and the reference image acquired by shifting the image in the substantially same direction as that in which the first and second openings of the light passing portion are arranged, while a shifted amount is varied by a predetermined amount.
The apparatus may further comprise: a second optical image forming unit which image forms the object; a second light receiving unit which image-forms the object by the second optical image forming unit; and a control unit which controls at least one of focusing and aperture of the second optical image forming unit, and exposure time of the second light receiving unit, based on the distance calculated by the distance calculating unit.
The apparatus may further comprise: a second light receiving unit which image-forms the object by the first optical image forming unit; and a control unit which controls at least one of focusing and aperture of the first optical image forming unit, and exposure time of the second light receiving unit, based on the distance calculated by the distance calculating unit.
Moreover, the apparatus may further comprise a recoding unit which records an image imaged by the second light receiving portion and the distance calculated by the distance calculating unit.
Moreover, the apparatus may further comprise: a drive unit which moves the light passing portion out of an optical path defined by a space such that the object is light-received by the light receiving unit; and a control unit which controls the optical image forming unit or the light receiving unit utilizing the distance calculated by the distance calculating unit.
The light passing portion may further include: a third opening which irradiates to the light receiving unit the light having passed through a third region in the optical image forming unit; a control unit which controls the optical image forming unit or the light receiving portion based on the distance calculated by the distance calculating unit; and a drive unit such that, when imaging a stereoscopic image of the object, while the third opening is being closed and the first and second openings of the light passing portion are opened, an image having passed through the first opening and an image having passed through the second opening are simultaneously imaged, and, when imaging the object, while the third opening of the light passing portion is being opened, an image having passed through is imaged by the light receiving unit.
Preferably, the first and second openings of the light passing portion are provided in the peripheral vicinity of the optical image forming unit, the third opening of the light passing portion is provided in a region including an optical axis of the optical image forming unit, and the third opening occupies larger region than that occupied by the first and second openings.
According of still another aspect of the present invention there is provided a stereoscopic imaging method for imaging a plurality of stereoscopic images acquired when an object is viewed from different points, the method comprising: picking up an image of the object at a light receiving unit via a light passing portion which restricts a passage of light in a region other than the different points; extracting an edge image of the image imaged in the light receiving unit; calculating correlation between the edge image of a specific region and a reference image acquired by shifting the edge image in a stereoscopic direction connecting the different points while a shifted amount is varied by a predetermine amount; obtaining a stereoscopic amount of the edge image of the specific region and the reference image based on the shifted amount at which the level of the correlation becomes minimum; and calculating a distance between an optical image forming unit and the object imaged at the specific region.
The method may further comprise: adjusting an imaging condition based on the calculated distance; removing a restriction set forth at the step of picking up the image; and imaging the object under the adjusted imaging condition.
This summary of the invention does not necessarily describe all necessary features of the present invention. The present invention may also be a sub-combination of the above described features. The above and other features and advantages of the present invention will become more apparent from the following description of embodiments taken in conjunction with the accompanying drawings.